Open peer review

Hello, and welcome to this short talk about Open Peer Review. We will begin by describing some “open” alternatives to traditional scholarly peer review practices. Since open peer review isn’t always an option, we mention some other further alternatives in a similar spirit. We touch on the potential advantages and limitations of open peer review, and give some examples of how it is used alongside other open research practices. The talk concludes with some suggestions for how you can try it for yourself.

Operationaly, open peer review covers a range of models. The name implies that scholarly peer review processes are not open by default. Let’s begin by reviewing the typical state of affairs. Fundamental to the traditional peer review model is the fact that:

the identities of at least one of the parties in the review process are not disclosed

the reviews themselves are not publicly available

Accordingly, open peer review reverses one or both of these conditions, and opens up the peer review process through dis-closure. Along the way, additional other aspects of the review process may also be made open to scrutiny.

This depends on how it is implemented: open peer review can involve sharing the identities of reviewers with authors, or publishing reviews alongside an article. This may expand to include ongoing open/online dialogue about a published article. A journal might introduce it across several steps. The British Medical Journal (BMJ) for example, started its journey towards open peer review by sharing the names of reviewers with authors, and now publishes the full pre-publication history of articles alongside them, including reviewer comments.

Advocates of open peer review practices contend that they “enhance the quality, reliability and effectiveness of the peer review process” and simultaneously “enhance the visibility, recognition and reputation of reviewers”.

Arguments have also been put forward that say that open peer review isn’t always the best strategy. Detractors of the practice suggest that reviewers may be less critical in an open review process, because this could expose the critics to criticism themselves if it turns out they are mistaken — or, even worse, they might be subject to reprisals even if they are correct. When open peer review was introduced in BMJ, the editor wrote “we hope our small move will contribute to a broader culture change.” By and large this change has not yet spread to the high-stakes world of grant funding decisions: applicants may know who was on a given review panel, but have no idea who said what inside the panel.

Open peer review isn’t universally adopted, but it can be seen within a context of related scholarly practices, which open up other aspects of scholarly communication. Some scholarly communities publish “commentaries” or “critiques” alongside a “target” or “plenary” article. The journal may publish the authors’ responses to those commentaries, effectively transforming a scholarly journal into an open discussion for a particular purpose. (Journals vary as to whether commentary articles are themselves peer reviewed.) Some review communities put a system of rebuttals in place in the review stage: this allows people who receive reviews to respond to the reviewer, although the reviewer remains anonymous.

Open peer review can be used together with other related open practices. For example, Octopus is “a new publishing platform for scholarly research”, created with support from JISC. When publishing on Octopus, rather than appearing all at once, the various sections of a research paper are uploaded by the author incrementally, as the work develops. Each of these segments can then be peer reviewed by any other Octopus user. Although there’s no guarantee from the people who run Octopus that submitted contents will be reviewed, any contributed reviews are ‘open’ insofar as both the content and identity of the reviewer are shared publicly. In this connection, there are related practices which do come with the guarantee of a review, as described in the companion video on Open Protocols. And there are further variations: for instance, a so-called “overlay journal” is built by combining articles hosted on a preprint repository with additional structure hosted elsewhere, which collects and may also openly publish peer reviews. (Incidentally, one thing you should not do is publish any anonymous reviews that you’ve received from a journal without permission: they remain copyrighted material, even though they are anonymous!)

A practical next step would be to find a journal (or another venue) in your research area that employs open peer review, and try it out for yourself! openreview.net is a free website that runs a reviewing system that quite a few conferences use to collect and share open reviews.

⚠ Practice Example

At Brookes, you may be able to help with the internal review of grant proposals (as a matter of policy this applies to grants with value above a certain threshold). In this collegial setting, the reviewer’s identity and constructive comments are shared with the proposer, but not made fully public. Contact your research office to find out how you can get involved in the grant review process.

Review of Corneli's Method for using Octopus to evaluate Open Research training

Pros and cons of open peer review. (1999). In Nature Neuroscience (Vol. 2, Issue 3, pp. 197–198). Springer Science and Business Media LLC. https://doi.org/10.1038/6295

Smith, R. (1999). Opening up BMJ peer review. In BMJ (Vol. 318, Issue 7175, pp. 4–5). BMJ. https://doi.org/10.1136/bmj.318.7175.4

CC BY-SA 4.0 Joe Corneli. Last modified: January 14, 2025. Website built with Franklin.jl and the Julia programming language.

Open peer review

What is open peer review?

Advantages of open peer review

Limitations of open peer review

Related practices

Open peer review within open research ecosystems

You can try it for yourself.